Proto-symbol emergence

Robotics can serve as a testbed for cognitive theories. One behavioral criterion for comparing theories is the extent to which their implementations can learn to exploit new environmental opportunities. Furthermore, a robotics testbed forces researcher to confront fundamental issues concerning how internal representations are grounded in activity. In our approach, a mobile robot takes the role of a creature that must survive in an unknown environment. The robot has no a priori knowledge about what constitutes a suitable goal | what is edible, inedible, or dangerous | or even its shape or how its body works. Nevertheless, the robot learns how to survive. The robot does this by tracking segmented regions of its camera image while moving. The robot projects these regions into a canonical wavelet domain that highlights color and intensity changes at various scales. This reveals sensory invariance that is readily extracted with Bayesian statistics. The robot simultaneously learns an adaptable sensorimotor mapping by recording how motor signals transform the locations of regions on its camera image. The robot learn about its own physical extension when it touches an object. But it also undergoes an internal state change analogous to the thirst quenching or nausea producing e ects of intake in animals. This allows the robot to learn what an object a ords | is it edible or poisonous? | by relating these e ects to learned clusters of invariance. In this way primitive symbols emerge. These proto-symbols provide the robot with goals that it can achieve by using its sensorimotor mapping to navigate, for example, toward food and away from danger. 1 The Motivation for Symbol Emergence Even one-celled animals are able to make distinctions, detecting the presence or absence of light or chemicals so as to climb a gradient to a food source. But whereas every other species lives in its niche, Homo sapiens alone live in a world of their own making. They adapt to the environment by adapting the environment to themselves. They can disembed themA grant from crest of jst and a grant from the Ministry of Education of Japan (No. 12750215) help support this research. Java is a trademark of Sun Microsystems. y Yoji Miyazaki is now at NEC, Corporation. z Masanao Koeda is now a Master's student at Nara Institute of Science and Technology. sensorimotor mapping motor planning direct perception actuators internal model outer environment motivation system affective appraisal automatic analog processes well learned skills are automated in the lower module recognition & attention decision environment body sensors deliberate abstract conscious Figure 1: The design for ro (from psyche + robot), a robot that exploits a ordances. selves, shift perspectives, stand back from the here and now, and ponder futures unseen. These abilities are often associated with language and especially the power of a word or symbol to stand in for something that is absent. Though the warning call of a vervet monkey can stand in for the perception of a predator [4], language is more than that. It is a kind of system of distinctions or di erences [29]. While as practised it has aspects that are messy and probabilistic, its syntax is for the most part productive, systematic, and inferentially coherent [7]. Inferential coherence refers to the fact that operations can be performed on sentences in such a way that, if the original sentences correspond to true states of a airs (e.g., Socrates is a man; and all men are mortal), the resulting sentences or conclusions also correspond to true states of a airs (Socrates is mortal). Of course, since Aristotle invented the syllogism, it has been known that a representation's syntax can encode its role in inference. What is di erent today is that we have computers capable of automating this process. This has strengthened the view that the mind | like language | is a kind of symbol system [26]. The traditional AI approach to constructing a symbol system involves a programmer determining a set elementary symbols and rules for combining and manipulating them [19, 30]. The symbols may be manipulated deductively [20] or procedurally [6]. In the latsegment regions no regions? scale (64 by 64) wavelet transform signature categorization and affective appraisal affective field select action (if stuck, avoid revisiting) wander internal state Y N match with invariance cluster and update Figure 2: ro produces compact wavelet signatures that correspond to segmented regions. The robot learns invariance clusters that match against these regions. Making contact with a region that corresponds to a certain kind of object (e.g., edible, poisonous, dangerous) results in an internal state change. The robot identi es this change with its matching clusters. Thus, proto-symbols emerge as the robot learns to categorize objects by their internal e ects (and visual similarities). The robot appraises recognized objects in terms of its current internal state (e.g., hungry, horny, bored). In this sense they correspond to di erent kinds of a ordances. Their relative location and hedonic value (given the robot's current internal state) determine a hydrodynamic potential eld in the robot's sensorimotor phase space, which the robot attempts to minimize by moving toward desirable objects while avoiding obstacles and dangerous objects (e.g., a predator or a cli ). ter case, operators transform state descriptions. Thus, the application of an operator is meant to be analogous to taking an imagined action during planning. Although it is possible to embody a symbol system in a robot so that objects instantiate symbols and symbolic operations cause the robot to act on those objects, this is a rather weak form of embodiment. This is because a symbol system's symbol manipulation obeys symbol-rule relations that are a priori and internal to the system. The \system of distinctions" may set up an in nite space of possibilities, but the distinctions and the possibilities they circumscribe are xed. But it has become increasingly clear to even proponents of symbol systems that symbol-object relations must be brought to bear on symbol manipulation [8, 11, 12, 16]. These relations are sensorimotor; they depend on having a particular body, and they change as that body or its environment changes. Sensorimotor relations in uence what the environment affords | what Gibson called a ordances [9] | and the kinds of sensory invariance on which any distinction can be made. Thus, they constrain not only what can be done but what can be perceived. Taken together Figure 3: A 2D-to-1D mapping with partition nets after only a few second of learning. The coordinates vary from 0 to 1. Circles represent data points. All points lie on f . The length of the line above or below a circle gives the prediction error f g. with cognitive and perceptual limitations, sensorimotor relations delimit the space of potential distinctions, and it is within this space that even the most abstract forms of reasoning must occur. To take account of these facts in developing reasoning machines, we propose an approach to emerging symbols from the bottom up. A mobile robot learns a sensorimotor model from experience so that it can make predictions about the consequences of its actions (x2). This involves learning to recognize a ordances (x3) and the actions needed to exploit them. For a fairly low-level task like robot navigation, this may work out to knowing what is where. But notions of object and distance must be discovered by the robot itself based on how its motor signals transform both internal indicators of well-being and the information that external objects project on to its camera image planes and other sensors [15]. Once a robot has learned to recognize a ordances, it can use its sensorimotor model to obtain goals. 2 Sensorimotor Learning: Partition Nets A robot may need to respond immediately to unexpected changes in its sensorimotor relations. Unlike neural networks, closest point methods are capable of learning immediately from single instances. However, they may give slower predictions and poorer generalizations, especially in high dimensional phase spaces. Since thousands of predictions may be required in the course of making a single plan and prediction errors can easily accumulate, we need an algorithm that can combine the strengths of closest point algorithms and neural networks. Partition nets [17] are an e cient on-line learning algorithm that can make fast predictions about well-practised movements while quickly adapting to changes in sensorimotor relations. The algorithm is interesting from a cognitive standpoint because it mirrors certain aspects of how humans learn. For example, in learning to ride a bike, we progress from reason-